The present invention relates to a data transmission method, a data transmission system, a data receiving method, and a data receiving apparatus and, more particularly, to a method for transmitting image data of MPEG4, through the Internet, according to a transmission mode adapted to the type of the image data.
In recent years, we have greeted the age of multimedia in which audio, video, and other data are integrally handled, and the conventional information media, i.e., means for transmitting information to men, such as newspapers, magazines, televisions, radios, and telephones, have been adopted as the targets of multimedia. Generally, xe2x80x9cmultimediaxe2x80x9d means media in which, not only characters, but also diagrams, speeches, and especially images are simultaneously expressed in relation with each other. In order to handle the conventional information media as the targets of multimedia, it is necessary to represent the data in digital formats.
When the quantity of data possessed by each of the above-described information media is estimated as the quantity of digital data, in the case of characters, the data quantity per character is only 1xcx9c2 byte. However, in the case of speech, the data quantity is 64 kbits per second (quality for telecommunication). Further, in the case of moving picture, the required data quantity is more than 100 Mbits per second (quality for current television broadcasting). So, in the above-described information media, it is not practical to handle such massive data as it is in the digital format. For example, although visual telephones have already been put to practical use by the ISDN (Integrated Services Digital Network) having a transmission ratio of 64 kbpsxcx9c1.5 Mbps, it is impossible to transmit an image from a television camera as it is by the ISDN.
So, data compression techniques are demanded. For example, for visual telephones, the moving picture compression techniques based on the H.261 and H.263 standards which have been standardized by ITU-T (International Telecommunication Unionxe2x80x94Telecommunication Sector) are employed. Further, according to the data compression technique based on the MPEG1 standard, it is possible to record image data as well as audio data in an ordinary music CD (compact disk).
MPEG (Moving Picture Experts Group) is an international standard relating to a data compression technique for an image signal corresponding to a moving picture, and MPEG1 is the standard for compressing moving picture data to 1.5 Mbps, i.e., data of a television signal to about {fraction (1/100)}. Since the transmission rate of the targets to which the MPEG1 standard is directed is limited to about 1.5 Mbps, in MPEG2 which has been standardized to meet the demand for higher image quality, moving picture data is compressed to 2xcx9c15 Mbps.
Furthermore, under the existing circumstances, standardization of MPEG4 is now proceeded by the working group for standardization of MPEG1 and MPEG2 (ISO/IEC JTC1/SC29/WG11), and this MPEG4 enables coding and signal processing in object units, and hereby realizes new functions required in the age of multimedia.
FIGS. 7, and 8(a)-8(d) are diagrams for explaining the object-by-object coding process.
In MPEG4, an image G corresponding to one frame shown in FIG. 7 is treated as a composite image which is obtained by compositing plural objects. The image G is composed of a background B (FIG. 8(a)), a big fish F1 as a first foreground (FIG. 8(b)), a small fish F2 as a second foreground (FIG. 8(c)), and a seaweed F3 as a third foreground (FIG. 8(d)).
Further, in the object-by-object coding process based on MPEG4, image data corresponding to the respective objects constituting the composite image G (the background B and the first to third foregrounds F1xcx9cF3) are encoded object by object. Then, coded image data (object data) corresponding to the respective objects are transmitted through a transmission medium.
On the other hand, in the object-by-object decoding process based on MPEG4, the coded image data corresponding to the respective objects are received object by object or in the multiplexed state, through the transmission medium. The received coded image data are decoded object by object, thereby generating decoded image data corresponding to each object. Then, the decoded image data corresponding to the respective objects are composited, thereby generating reproduced image data (scene data) corresponding to the composite image (decoded and reproduced image) G.
In the above-described process for transmitting the object-by-object coded image data, not only the coded image data (object data) obtained by coding the image data object by object but also control information are transmitted through the transmission medium. The control information includes, for example, scene description information which indicates the locations of the respective objects in one frame for compositing and displaying the objects, i.e., the display area of the composite image (refer to FIG. 15(b)).
Meanwhile, in recent years, a video distribution system has spread, in which the user can gain access to his/her favorite moving picture through a computer network.
FIG. 9 is a schematic diagram for explaining such video distribution system.
In a video distribution system 700, a plurality of networks 701, 702 and 703 are connected to each other, and a plurality of servers 705, 706 and 707 which distribute video information are connected to the networks 701, 702 and 703, respectively. Further, a plurality of video reception and reproduction terminal units (hereinafter referred to simply as xe2x80x9cterminal unitsxe2x80x9d) which receive the distributed video information are connected to the respective networks 701xcx9c703. To simplify the description, only a terminal unit 704 connected to the network 701 is shown in FIG. 9.
In the video distribution system 700 so constructed, when the terminal unit 704 receives video information distributed from the server 705, initially, the terminal unit 704 contacts the server 705, and effectuates a circuit connection with the server 705. Thereafter, the terminal unit 704 receives coded image data distributed from the server 705, and reproduces the image data by decoding.
Hereinafter, a description will be given of a specific process of obtaining predetermined image data through the Internet as the above-described network.
As described above, a plurality of information sources (servers) are connected to the Internet which forms the network in the current information distribution system, and the respective information sources contain, for example, information of home pages relating to various subjects. Now it is assumed that the terminal unit 704 is connected to an information source having information of a home page HP the title of which is xe2x80x9cWorld of Dinosaursxe2x80x9d, and this home page HP is displayed on a display of the terminal unit 704.
In this home page HP, as shown in FIG. 10, items of video scenes relating to dinosaurs are represented by character strings xe2x80x9cscene 1xe2x80x9dxcx9cxe2x80x9cscene 3xe2x80x9d, and the display areas of these character strings are designation areas D1xcx9cD3 for designating the moving pictures. In this state, the user moves the mouse pointer MP to the designation area D1 corresponding to the character string xe2x80x9cscene 1xe2x80x9d in the home page HP and clicks the mouse, an image MPs of a video scene P linked to the character string xe2x80x9cscene 1xe2x80x9d is displayed as shown in FIG. 11(a).
As a data transmission method for transmitting the image (text, audio, and video) data on the Internet, download type transmission and stream type transmission are currently employed.
In the download type transmission, a video file (image data) transmitted from a distribution server is once copied at the terminal and, thereafter, the image corresponding to the video file is reproduced. So, the terminal cannot start image reproduction until the file transmission is completed. That is, there is a latency time for transmission and, therefore, the download type transmission is not suitable for long-hours reproduction of video and audio. On the other hand, in the stream type transmission, while transmitting video data or the like from the distribution server to the terminal, image reproduction is carried out on the basis of the received data at the terminal end.
Therefore, data to be processed in real time, such as video data and audio data, are transmitted by the stream type transmission.
Recently, stream type transmission using a protocol called RTP (Real-time Transport Protocol) has been mainly employed. In a communication system in which a distribution server (transmitting end) S and a terminal (receiving end) T such as a personal computer are connected by a circuit such as ISDN in the Internet as shown in FIG. 16, transmission of image data is performed according to the RTP.
In the data transmission according to the RTP, processes for the respective packets are synchronized between the transmitting end and the receiving end by using time stamps as time information, and asynchronous (late arrival) packets and error packets in which transmission errors have occurred are discarded at the receiving end. Further, discarded or lost packets are detected at the receiving end, according to the absences of sequence numbers given to these packets.
Under the circumstances described above, in recent years, a method for transmitting image data of MPEG4 through the Internet has been examined.
In MPEG4, the video scene P shown in FIG. 11(a) is treated as a composite image G1 which is composed of four objects ob1, ob2, ob3, and ob4 (refer to FIG. 11(b)). That is, in this composite image G1, the object ob1 is a background as a still picture showing the sky or the like (FIG. 12(a)), the object ob2 is a first foreground as a cyclic moving picture showing a volcano (FIG. 12(b)), the object ob3 is a second foregound as a moving picture showing a big dinosaur (FIG. 12(c)), and the object ob4 is a third foreground as a moving picture showing a small dinosaur (FIG. 12(d)).
Hereinafter, a data transmission method based on MPEG4, which is currently examined, will be described by using FIGS. 15(a) and 15(b). In the following description, a plurality of servers may be connected to the terminal T.
In this data transmission method, image data corresponding to one scene (composite image) treated in MPEG4 is obtained for each of plural objects constituting the scene, from a predetermined server, by the RTP, through the Internet.
Initially, as already described with respect to FIG. 10, in the state where the home page HP is displayed on the display unit of the terminal T, when the user clicks the mouse at the designation area D1 corresponding to the character string xe2x80x9cscene 1xe2x80x9d, a session description protocol (SDP) (FIG. 13(a)) and an initial object descriptor (IOD) (FIG. 13(b)), which correspond to the scene (composite image) G1 and are linked to the character string xe2x80x9cscene 1xe2x80x9d, are transmitted from the server S containing these data to the terminal T through the Internet (refer to FIG. 9).
With reference to FIG. 13(a), in the SDP (hereinafter also referred to as SDP information), the type of data is described in association with a transmission mode and a URL (Uniform Resource Locator) of the data, which are required for reproduction of the composite image. To be specific, as the above-mentioned data, there are not only the object data (entity data) for reproducing the respective objects constituting the scene, i.e., image data, audio data, and text data, but also control information, i.e., an initial object descriptor (IOD), an object descriptor stream (ODS), and a scene description stream (SDS).
Further, as shown in FIG. 13(b), in the initial object descriptor (hereinafter also referred to as IOD information), data of the object descriptor (OD) corresponding to the ODS itself, and data of the object descriptor (OD) corresponding to the SDS itself, are stored.
Further, as shown in FIG. 14(a), in the ODS (hereinafter also referred to as ODS information), data of object descriptors (OD) of all objects constituting the scene are stored. Further, as shown in FIG. 14(b), in the SDS (hereinafter also referred to as SDS information), data of scene descriptions SD1xcx9cSDn corresponding to predetermined times t1xcx9ctn are stored.
When the SDP information and the IOD information are transmitted to the terminal T, the terminal T requests the server S to transmit the ODS information corresponding to the respective objects (i.e., object descriptors OD1xcx9cOD4) and the entity data, by specifying their uniform resource locators (URL) and synchronization sources (hereinafter referred to simply as SSRC). Further, the terminal T requests the server S to transmit the SDS information corresponding to the scene G1 (i.e., the scene descriptions SD1xcx9cSDn), by specifying the URL and SSRC. The SSRC is a channel ID for identifying a channel of the RTP packet.
On receipt of the request from the terminal T, the server S packetizes the requested data, gives the SSRC specified by the terminal to each packet, and transmits the packets to the terminal T according to the RTP.
The terminal T receives the image data (object data) of the respective objects constituting the scene G1, the object descriptors OD1xcx9cOD4, and the scene descriptions SD1xcx9cSDn. Based on these data, the terminal T reproduces and displays the image of the scene G1.
FIG. 15(a) shows the specific contents of the object descriptor OD1 (id=100), and FIG. 15(b) shows the specific contents of the scene description SD1 (time t1).
However, the image data according to MPEG4 includes data (control information) which prevents image reproduction if a transmission error occurs in this data, such as the SDP information, the IOD information, the SDS information, and the ODS information. When such data (control information) is transmitted according to the RTP, since recovery against the transmission error is not performed, the state where the image data cannot be reproduced, occurs frequently at the receiving end.
Further, when download type transmission and the stream type transmission are performed in arbitrary order, the image data are sometimes transmitted by the download transmission while reproducing the image data transmitted by the stream type transmission. In this case, the data transmission quantity during scene reproduction increases significantly, and the consumption of the transmission band of the network cannot be minimized.
Further, although the number of the objects constituting he scene and the hierarchy of the objects can be grasped from the SDS information and the ODS information, the correspondence between the address (URL) of the object data (image data corresponding to each object) and each object cannot be obtained.
The present invention is made to solve the above-described problems and has for its object to provide a data transmission method, a data transmission system, a data receiving method, and a data receiving apparatus, which avoid unreliability of image reproduction at a receiving terminal due to transmission error, and assure reliable image reproduction at the receiving terminal even when some transmission error occurs.
It is another object of the present invention to provide a data transmission method, a data transmission system, a data receiving method, and a data receiving apparatus, which prevent data transmission quantity during from increasing considerably scene reproduction, and thereby minimize the consumption of the transmission band of the network.
It is still another object of the present invention to provide a data transmission method, a data transmission system, a data receiving method, and a data receiving apparatus, in which each object can be associated with the URL (Uniform Resource Locator) of image data corresponding to the object, on the basis of the SDP (Session Description Protocol) corresponding to the scene.
Other objects and advantages of the invention will become apparent from the detailed description that follows. The detailed description and specific embodiments described are provided only for illustration since various additions and modifications within the scope of the invention will be apparent to those of skill in the art from the detailed description.
According to a first aspect of the present invention, there is provided a data transmission method of transmitting plural pieces of object data for reproducing plural objects constituting a scene, from the transmitting end to the receiving end. This method comprises a first transmission process of transmitting first object data, among the plural pieces of object data, such that reproduction of the scene is performed at the receiving end on the basis of the first object data after the transmission of the first object data has been completed; and a second transmission process of transmitting second object data other than the first object data, among the plural pieces of object data, such that reproduction of the scene is performed at the receiving end on the basis of the second object data during the transmission of the second object data. Therefore, extreme increase in the data transmission quantity during scene reproduction is avoided, and the consumption of the transmission band of the network is minimized.
According to a second aspect of the present invention, there is provided a data transmission method for transmitting scene data for reproducing a scene, from the transmitting end to the receiving end. This method comprises a first transmission process of transmitting first data which is a part of the scene data, in accordance with a first transmission protocol which does not necessarily perform retransmission against transmission errors; and a second transmission process of transmitting second data which is a part of the scene data, in accordance with a second transmission protocol which performs retransmission against transmission errors. The second data is scene description information and object relevant information, the scene description information indicating the hierarchy of the respective objects constituting the scene, by object identifiers which are given to the respective objects for identifying these objects, and the object relevant information indicating additional information relating to the respective objects, in association with the object identifiers of the respective objects. Therefore, this method can prevent image reproduction at the receiving end from becoming unreliable due to transmission error, and the receiving end can perform reliable image reproduction even when some transmission error occurs.
According to a third aspect of the present invention, there is provided a data transmission method for transmitting scene data for reproducing a scene, from the transmitting end to the receiving end. This method comprises the steps of transmitting object data for reproducing plural objects constituting the scene, object by object, from the transmitting end to the receiving end; and transmitting table information from the transmitting end to the receiving end, the table information associating an object identifier given to each object for identifying the object, with location information indicating the location of object data corresponding to the object. Therefore, the receiving end can associate each object with the location of object data corresponding to the object, on the basis of the table information.
According to a fourth aspect of the present invention, there is provided a data transmission system comprising a data transmission apparatus for transmitting plural pieces of object data for reproducing plural objects constituting a scene; and a data receiving apparatus for receiving the plural pieces of object data and reproducing the scene on the basis of the object data. The data transmission apparatus comprises a first transmission unit for transmitting first object data, among the plural pieces of object data, such that reproduction of the scene is performed at the receiving end on the basis of the first object data after the transmission of the first object data has been completed; and a second transmission unit for transmitting second object data other than the first object data, among the plural pieces of object data, such that reproduction of the scene is performed at the receiving end on the basis of the second object data during the transmission of the second object data. Therefore, extreme increase in the data transmission quantity during scene reproduction is avoided, and the consumption of the transmission band of the network is minimized.
According to a fifth aspect of the present invention, in the data transmission system of the fourth aspect, the data transmission apparatus transmits, by using the first transmission unit, control information for controlling transmission of the object data and reproduction of the objects, as initial data to be transmitted to the receiving end in the transmission process for data corresponding to one scene, the control information including transmission mode identification information indicating that the object data corresponding to each of the objects constituting one scene is to be transmitted by the first transmission unit or the second transmission unit. Therefore, in addition to the effect of minimizing the consumption of the transmission band of the network, this system can prevent image reproduction at the receiving end from becoming unreliable due to transmission error.
According to a sixth aspect of the present invention, there is provided a data transmission system comprising a data transmission apparatus for transmitting scene data for reproducing a scene; and a data receiving apparatus for receiving the scene data and reproducing the scene. The data transmission apparatus comprises a first transmission unit for transmitting first data which is a part of the scene data, in accordance with a first transmission protocol which does not necessarily perform retransmission against transmission errors; and a second transmission unit for transmitting second data which is a part of the scene data, in accordance with a second transmission protocol which performs retransmission against transmission errors. The second transmission unit transmits, as the second data, scene description information and object relevant information, the scene description information indicating the hierarchy of the respective objects constituting the scene, by object identifiers which are given to the respective objects for identifying these objects, and the object relevant information indicating additional information relating to the respective objects, in association with the object identifiers of the respective objects. Therefore, this system can prevent image reproduction at the receiving end from becoming unreliable due to transmission error, and the receiving end can perform reliable image reproduction regardless of some transmission error.
According to a seventh aspect of the present invention, there is provided a data transmission system comprising a data transmission apparatus for transmitting, object by object, object data for reproducing plural objects constituting a scene; and a data receiving apparatus for receiving the object data and reproducing the scene on the basis of the object data. The data transmission apparatus comprises an information transmission unit for transmitting table information which associates an object identifier given to each object for identifying the object, with location information indicating the location of object data corresponding to the object. Therefore, the receiving end can associate each object with the location of object data corresponding to the object, on the basis of the table information.
According to an eighth aspect of the present invention, in the data transmission system of the seventh aspect, the information transmission unit transmits the table information in such a manner that the table information is included in control information for controlling transmission of the object data and reproduction of the objects, the control information being initial information to be transmitted to the receiving end in the transmission process for data corresponding to the scene. Therefore, the receiving end can associate each object with the location of object data corresponding to the object, on the basis of the table information.
According to a ninth aspect of the present invention, in the data transmission system of the seventh aspect, the information transmission unit transmits the table information in such a manner that the table information is included in object relevant information which indicates additional information relating to the respective objects in association with object identifiers of the respective objects. Therefore, the receiving end can associate each object with the location of object data corresponding to the object, on the basis of the table information.
According to a tenth aspect of the present invention, there is provided a data receiving method for receiving plural pieces of object data for reproducing plural objects constituting a scene, transmitted from the transmitted end, and reproducing the scene on the object data. This method comprises a first receiving process of receiving first object data among the plural pieces of object data, and reproducing the scene on the basis of the first object data after the reception of the object data has been completed; and a second receiving process of receiving second object data other than the first object data among the plural pieces of object data and, simultaneously reproducing the scene on the basis of the second object data. Therefore, extreme increase in the data transmission quantity during scene reproduction is avoided, and the consumption of the transmission band of the network is minimized.
According to an eleventh aspect of the present invention, there is provided a data receiving method for receiving scene data for reproducing a scene, transmitted from the transmitting end, and reproducing the scene. This method comprises a first receiving process of receiving first data which is a part of the scene data, transmitted according to a first transmission protocol which does not necessarily perform retransmission against transmission errors; and a second receiving process of receiving second data which is a part of the scene data, transmitted according to a second transmission protocol which performs retransmission against transmission errors. The second data is scene description information and object relevant information, the scene description information indicating the hierarchy of the respective objects constituting the scene, by object identifiers corresponding to the respective objects, and the object relevant information indicating additional information relating to the respective objects, in association with the object identifiers of the respective objects. Therefore, this method can prevent image reproduction at the receiving end from becoming unreliable due to transmission error, and satisfactory scene reproduction is performed regardless of some transmission error.
According to a twelfth aspect of the present invention, there is provided a data receiving method for receiving scene data for reproducing a scene, transmitted from the transmitting end, and reproducing the scene. This method comprises the steps of receiving object data for reproducing plural objects reconstituting the scene, which are transmitted object by object from the transmitting end; and receiving table information transmitted from the transmitting end, the table information associating an object identifier given to each object for identifying the object, with location information indicating the location of object data corresponding to the object. Therefore, during scene reproduction, each object can be associated with the location of object data corresponding to the object, on the basis of the table information.
According to a thirteenth aspect of the present invention, there is provided a data receiving apparatus for receiving plural pieces of object data for reproducing plural objects constituting a scene, transmitted from the transmitting end, and reproducing the scene on the basis of the object data. This apparatus comprises a first receiving unit for receiving first object data among the plural pieces of object data; a second receiving unit for receiving second object data other than the first object data among the plural pieces of object data; and a reproduction unit for reproducing the scene on the basis of the first object data and the second object data, in which scene reproduction based on the first object data is performed after reception of the first object data has been completed, while scene reproduction based on the second object data is performed in parallel with the reception of the second object data. Therefore, extreme increase in the data transmission quantity during scene reproduction is avoided.
According to a fourteenth aspect of the present invention, in the data receiving apparatus of the thirteenth aspect, control information for controlling transmission of the object data and reproduction of the objects is received by the first receiving unit, as initial data transmitted to the receiving end in the transmission process for data corresponding to the scene; and the object data corresponding to each object is received by one of the first and second receiving units, according to transmission mode identification information which is included in the control information and indicates that the object data is to be received by the first receiving unit or the second receiving unit. Therefore, in addition to the effect of minimizing the consumption of the transmission band of the network, this apparatus can prevent image reproduction at the receiving end from becoming unreliable due to transmission error.
According to a fifteenth aspect of the present invention, there is provided a data receiving apparatus for receiving scene data for reproducing a scene, transmitted from the transmitting end, and reproducing the scene. This apparatus comprises a first receiving unit for receiving first data which is transmitted as a part of the scene data in accordance with a first transmission protocol that does not necessarily perform retransmission against transmission errors; a second receiving unit for receiving second data which is transmitted as a part of the scene data in accordance with a second transmission protocol that performs retransmission against transmission errors. The second receiving unit receives, as the second data, scene description information and object relevant information, the scene description information indicating the hierarchy of the respective objects constituting the scene, by object identifiers corresponding to the respective objects, and the object relevant information indicating additional information relating to the respective objects, in association with the object identifiers of the respective objects. Therefore, this apparatus can prevent image reproduction at the receiving end from becoming unstable due to transmission error.
According to a sixteenth aspect of the present invention, there is provided a data receiving apparatus for receiving object data for reproducing objects constituting a scene, transmitted object by object from the transmitting end, and reproducing the scene. This apparatus comprises a first receiving unit for receiving first data which is transmitted as a part of the scene data in accordance with a first transmission protocol that does not necessarily perform retransmission against transmission errors; and a second receiving unit for receiving second data which is transmitted as a part of the scene data in accordance with a second transmission protocol that performs retransmission against transmission errors. The second receiving unit receives, as the second data, table information transmitted from the transmitting end, the table information associating an object identifier given to each object for identifying the object, with location information indicating the location of object data corresponding to the object. Therefore, even when some transmission error occurs, each object can be associated with the location of object data corresponding to the object on the basis of the table information.
According to a seventeenth aspect of the present invention, in the data receiving apparatus of the sixteenth aspect, the second receiving unit receives the table information as information included in control information for controlling transmission of object data corresponding to the respective objects and reproduction of the objects, the control information being received first as the scene data corresponding to one scene. Therefore, even when some transmission error occurs, each object can be associated with the location of object data corresponding to the object on the basis of the table information.
According to an eighteenth aspect of the present invention, in the data receiving apparatus of the sixteenth aspect, the second receiving unit receives the table information as information included in object relevant information transmitted from the transmitting end, the object relevant information indicating additional information relating to the respective objects, in association with object identifiers of the respective objects. Therefore, even when some transmission error occurs, each object can be associated with the location of the object data corresponding to the object on the basis of the table information.